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In this technical report, we will make two observations concerning symmetries of the prob- 
ability distribution resulting from projection of a piece of p- dimensional data onto a random 
m-dimensional subspace of W, where m < p. In particular, we shall observe that such distri- 
butions are unchanged by reflection across the original data vector and by rotation about the 
original data vector. 

To start, let us introduce some notation. Suppose that x is the original piece of data in R p . 
We randomly generate m vectors e±, . . . ,e m e W from the Gaussian distribution with mean 
and covariance the identity matrix I pxp - We then form the matrix E e ]R pxm with columns 
ei, . . . , e m . The projection of x onto the m-dimensional subspace of W spanned by e 

1 j • • • j 6 m 

is then given by Px where P = E(E T E)' 1 E T . 

We can now prove the following two lemmas about the distribution of the random vector 
Px. The first shows that the distribution of Px is unchanged when reflected across x. The 
second shows that this distribution is unchanged when rotated about the axis of x. 

Lemma 1 (Symmetry of the distribution of Px under reflection across x). Suppose x is a 
fixed point in W and let Px be a random vector with P as defined above. Define the reflection 
operator P x as 

Px(y) = y + 2 ((y, x)x - y) = 2(y, x)x - y (1) 
where x = Then the distribution of Px is the same as the distribution o/P x (Px). 

Proof For every realization e°, . . . , e° t of the random variables ei, . . . , e m , there is an equally 
likely realization P x (e ( j ) ), . . . , P x (e^J. This can be easily seen from the fact that the Gaussian 
distribution Af(0, I pX p) is symmetric across any line through the origin of W. 

We will show that if we define P( e o e oj(x) as the projection of x onto the subspace spanned 
bye°,...,e^ Then 

^(i? x (e?),...,R x (eO l ))( x ) = #x(P(e?,...,eO,)( X ))- ( 2 ) 

That is, the projection of x onto the reflected vectors P x (e ( j ) ), . . . , P x (e^) is the reflection of 
that onto the original random vectors e°, ■ ■ ■ , e^. 

To show this, we first observe three properties of the operator P x : 

• Property 1: P x is a linear operator: 

P x (aa + /3b) = 2(aa + /3b, x)x — «a — /3b 

= a(2(a,x)x- a) + /3(2(b,x)x- b) 
= aP x (a) + /3P x (b) 

• Property 2: The operator P x preserves inner products (and hence norms as well): 

(P x (a),P x (b)) = (2(a,x)x-a,2(b,x)x-b) 

= 4(a,x)(b,x) -4(a,x)(b,x) + (a, b) 
= (a,b) 

• Property 3: For any orthonormal u l5 . . . , and any b, the projection of P x (b) onto 
Plx(ui), . . . , P x (u fc ) is the reflection of that of b onto u l5 . . . , u k : 
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Using the first two properties above, and the fact that Px(iii), . . . , P x (u fe ) must be or- 
thonormal by Property 2, we have: 

k 

(P x (b)) = ^(^(b),^^))^^) 

k 

= J?x ( P («i,-.-)( b )) 

Using the above three properties, we can easily see that if we perform Gram-Schmidt or- 
thogonalization on e°, . . . , to obtain ortho normalized vectors Ui, . . . , u m , then performing 
Gram-Schmidt orthogonalization on i? x (e5), . . . , P x (e^J must result in P x (ui), . . . , P x (u m ). To 
see this, we note that Gram-Schmidt involves two alternating steps: (i) we subtract from the 
currently selected vector its orthogonal projection onto those orthonormal vectors already ob- 
tained and (ii) we scale the resulting vector by 1 over its norm. Suppose that we start with the 
two sets of vectors e^, . . . , and P^e^), . . . , P x (e^). We note that the second set are initially 
the reflections of the first set. If we run the steps of Gram-Schmidt on the two sets of vectors 
simultaneously, then each step of Gram-Schmidt preserves the property that the second set of 
vectors are the reflections of the first set. In the case of step (i), the orthogonal projections that 
we subtract off from the second set are reflections by Property 3 above of those we subtract 
off from the corresponding vector in the first set. Then, the linearity of P x (Property 1 above) 
guarantees that the resulting difference vector in the second set is a reflection of that obtained 
for the first set. In the case of step (ii), the norms we divide by are equal (Property 2 above). 

Hence, we find that using Property 3 above and the fact that P x (x) = x, we have that: 

* > (Jk(e?) 1 ... 1 fl B (e°,))( x ) = %(ui) ft,W)W) = ^x(-P(ui,...,u m )(x)) = P^P^o,...^) (x)) 

Finally, since for every realization ej, . . . , of the random variables e±, . . . ,e m , resulting 
in the projection Px = P( e °,...,e°j( x )> there is an equally likely realization P x (e ( j ) ), . . . , P x (e^J, 
resulting in the projection P^eO),...,/?^))^) = Px(P(e ( 1 ',..., e o i )(x)) = P x (Px), we have that the 
probability distribution / of Px satisfies 

/(Px)</(P x (Px)). 

Similarly, since for every realization P x (e ( j ) ), . . . , P x (e^J, resulting in the projection P x (Px), 
there is an equally likely realization P x (P x (e ( j ) )), . . . , P x (P x (e^J) = e°, . . . , e^, resulting in the 
projection Px, we have that 

/(P x (Px))</(Px). 

These inequalities show that 

/(Px) = / (P x (Px)) . 

This proves Lemma 1. □ 
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Lemma 2 (Symmetry of the distribution of Px under rotation about x). Suppose x is a fixed 
point in W and let P be as defined above. Let V G R pxp be an orthogonal matrix with first 
column x = -j^-u and let 

||x|| 

V u (p-i)xi Q J 

where Q is in the special orthogonal group SO p _i, so that Q x represents an arbitrary rotation 
ofW about x. Then the distribution of Px is the same as the distribution o/Q x (Px). 

Proof The proof follows the exact same structure as that of Lemma 1. 

Similarly, we note that for every realization e^, . . . , of the random variables e±, . . . ,e m , 
there is an equally likely realization Q x e®, . . . , Q x e^, since the Gaussian distribution is rota- 
tionally symmetric. 

Then we would like to show that if we define P( e °,...,e° )( x ) as the projection of x onto the 
subspace spanned by e°, . . . , e^, then 

^(Q x e?,...,Q x eOJ ( x ) = <2xP( e ?,...,e°J ( x ) • (3) 

That is, the projection of x onto the rotated vectors Q x e®, . . . , Q^e 1 ^ is the rotation of that 
onto the original random vectors e°, . . . , e^. 

As before, to prove this, we first note that Q x is a linear operator. Q x also preserves inner 
products and norms (i.e. (Q x (sl), Q x (h)) = (a, b) for all a and b) since V is an orthogonal 
matrix and Q is in the special orthogonal group SO p -\. 

Using these two properties, we can show that for any vector b and any orthonormal set 
ui, . . . , Ufc, we have that 

k 

^((3iUi,...,Q x U)t) 

(Q x b) = ^2{Q x b,Q x Uj)Q x Uj 

i=i 

k 
3=1 




= Q> 

= Qx-P(ui,...,u fc )( b ) 

The same argument as before can be used with the above three properties to show if 
ui,...,u m is the result of Gram-Schmidt orthogonalization on the vectors e°,...,e^, then 
Qx u i, . . . , Q x u m must be the result of the Gram-Schmidt orthogonalization on <5x e ?, • • • , Q*. e m- 

Finally, using the above and the fact that Q x x. = x, we see that 

P (Q x (e«),...,Q x (e°J)( x ) = P (Qx( Ul ),...,Q x (u m )) (x) = <2 x P(ui,...,u m )( x ) = <2x( J P( e °,..,e° l )( X )) 

Since for every realization e?, . . . , of the random variables, ei, . . . , e m , resulting in the 
projection P( e o : ..^^(x), there is an equally likely realization <5 x (e?), . . . , Q x (e^J, resulting in 
the projection P(q x ( e o ),..., Q x ( e °j)( x ) — QxP( e 5,...,e° l )( x ) = QxPx, we see that the probability 
distribution / of Px satisfies 

/(Px) < /(Q x Px). 
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Moreover, noting that the matrix 

^L-U V') v,r 

has the same properties as Q x , we can see that for every realization Q x e^, . . . , Q x e^ of e 1; . . . , e m , 
resulting in the projection Q x Px, there is an equally likely realization Q^Q^e®, . . . , Q~ 1 Q x e m = 
e^, . . . , eP m , resulting in the projection Px. We therefore also have 

/(QxPx) < /(PX). 

These inequalities show that 

/(Px) = /(Q x Px). 

This proves Lemma 2. □ 
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